chore(config): bump hypershift operator digest (bisect e2e #5789)#5910
Merged
Conversation
|
Skipping CI for Draft Pull Request. |
Collaborator
Author
|
/hold |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR is a diagnostic bisect change intended to isolate whether the Hypershift operator image digest bump from PR #5789 is responsible for the CreateHCPCluster E2E provisioning timeouts. It updates only the Hypershift operator digest (keeping other component digests at main) and propagates that digest into the rendered dev configs used by CI and westus3 dev variants.
Changes:
- Bump the default Hypershift operator image digest in
config/config.yamltosha256:c22bdb0f…. - Update the corresponding rendered dev environment YAMLs (ci00/ci01 and westus3 dev/perf/pers/cspr) to use the same digest.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| config/config.yaml | Updates the default Hypershift operator image digest used for config materialization. |
| config/rendered/dev/pers/westus3.yaml | Propagates the Hypershift digest bump into rendered pers westus3 config. |
| config/rendered/dev/perf/westus3.yaml | Propagates the Hypershift digest bump into rendered perf westus3 config. |
| config/rendered/dev/dev/westus3.yaml | Propagates the Hypershift digest bump into rendered dev westus3 config. |
| config/rendered/dev/cspr/westus3.yaml | Propagates the Hypershift digest bump into rendered cspr westus3 config. |
| config/rendered/dev/ci01/centralus.yaml | Propagates the Hypershift digest bump into rendered CI01 centralus config. |
| config/rendered/dev/ci00/centralus.yaml | Propagates the Hypershift digest bump into rendered CI00 centralus config. |
Collaborator
Author
|
/unhold |
Collaborator
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: inbharajmani, raelga The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This was referenced Jul 3, 2026
openshift-merge-bot Bot
pushed a commit
that referenced
this pull request
Jul 6, 2026
Advances every component in config/config.yaml to its latest digest **except clusters-service**, which is intentionally pinned at the last-known-good digest `sha256:6a49b32…` (vcs-ref `18b5a25`, 2026-06-19). ## Why The automated bulk bump #5789 started failing `ci/prow/e2e-parallel` after the clusters-service image moved to `sha256:b17f6fe…` (vcs-ref `ee741db`). Bisecting the bump into per-component PRs isolated the culprit: - #5910 (hypershift only) — e2e-parallel green - #5911 (ACM/MCE only) — e2e-parallel green (merged) - #5920 (**clusters-service only**) — e2e-parallel **fails reproducibly** (2 consecutive runs) on the exact same specs: `test/e2e/complete_cluster_create_multiversion.go:172` "verify simple web app runs" for the **candidate 4.22 and 5.0** channels, with `route was never reachable: dial tcp 10.0.0.5:443: i/o timeout` — the control plane + node pools provision, but the data-plane ingress on the newest OCP channels never becomes reachable. 4.20/4.21 are unaffected. - #5912 (everything except CS) — only flaky/environmental failures (1 then 16, non-reproducible signature), consistent with the shared-CI ARM-throttling episode also hitting unrelated PRs (e.g. #5915, which edits only alert YAML). An image-digest bump is a behavioral change (the cluster runs the new build), and the CS-only PR reproduces a version-specific data-plane regression, so we pin CS at the good digest while letting the remaining components advance. Follow-up on the bad CS image is tracked in the Jira below. Jira: https://redhat.atlassian.net/browse/AROSLSRE-1395 ## Components bumped (clusters-service pinned, not bumped) | Component | Old | New | | --- | --- | --- | | acrpull | v0.1.23 | v0.1.24 | | arobit forwarder | v5.0.4 (06-19) | v5.0.4 (07-03) | | mdsd | 1.42.0-20260615 | 1.42.0-20260629 | | kube-events | 20260621.1 | 20260701.1 | | maestro (provider) | v1.8.2 (06-11) | v1.8.2 (06-26) | | hypershift | 9aeb1f3 | 488ef0e | | OADP velero-server | 1.6.1 (06-25) | 1.6.1 (07-02) | | OADP velero azure-plugin | 1.6.1 (06-25) | 1.6.1 (07-02) | | OADP velero hypershift-plugin | 1.6.1 (06-25) | 1.6.1 (07-02) | | kube-state-metrics | v2.19.0 (06-12) | v2.19.0 (06-30) | | maestro-agent-sidecar (nginx) | azl3.0.20260602 | azl3.0.20260616 | | image-sync/oc-mirror | 690892d | 5bfc996 | | **clusters-service** | **6a49b32 (pinned — good)** | **— (b17f6fe held back)** |
tmstff
pushed a commit
to tmstff/ARO-HCP
that referenced
this pull request
Jul 7, 2026
Advances every component in config/config.yaml to its latest digest **except clusters-service**, which is intentionally pinned at the last-known-good digest `sha256:6a49b32…` (vcs-ref `18b5a25`, 2026-06-19). ## Why The automated bulk bump Azure#5789 started failing `ci/prow/e2e-parallel` after the clusters-service image moved to `sha256:b17f6fe…` (vcs-ref `ee741db`). Bisecting the bump into per-component PRs isolated the culprit: - Azure#5910 (hypershift only) — e2e-parallel green - Azure#5911 (ACM/MCE only) — e2e-parallel green (merged) - Azure#5920 (**clusters-service only**) — e2e-parallel **fails reproducibly** (2 consecutive runs) on the exact same specs: `test/e2e/complete_cluster_create_multiversion.go:172` "verify simple web app runs" for the **candidate 4.22 and 5.0** channels, with `route was never reachable: dial tcp 10.0.0.5:443: i/o timeout` — the control plane + node pools provision, but the data-plane ingress on the newest OCP channels never becomes reachable. 4.20/4.21 are unaffected. - Azure#5912 (everything except CS) — only flaky/environmental failures (1 then 16, non-reproducible signature), consistent with the shared-CI ARM-throttling episode also hitting unrelated PRs (e.g. Azure#5915, which edits only alert YAML). An image-digest bump is a behavioral change (the cluster runs the new build), and the CS-only PR reproduces a version-specific data-plane regression, so we pin CS at the good digest while letting the remaining components advance. Follow-up on the bad CS image is tracked in the Jira below. Jira: https://redhat.atlassian.net/browse/AROSLSRE-1395 ## Components bumped (clusters-service pinned, not bumped) | Component | Old | New | | --- | --- | --- | | acrpull | v0.1.23 | v0.1.24 | | arobit forwarder | v5.0.4 (06-19) | v5.0.4 (07-03) | | mdsd | 1.42.0-20260615 | 1.42.0-20260629 | | kube-events | 20260621.1 | 20260701.1 | | maestro (provider) | v1.8.2 (06-11) | v1.8.2 (06-26) | | hypershift | 9aeb1f3 | 488ef0e | | OADP velero-server | 1.6.1 (06-25) | 1.6.1 (07-02) | | OADP velero azure-plugin | 1.6.1 (06-25) | 1.6.1 (07-02) | | OADP velero hypershift-plugin | 1.6.1 (06-25) | 1.6.1 (07-02) | | kube-state-metrics | v2.19.0 (06-12) | v2.19.0 (06-30) | | maestro-agent-sidecar (nginx) | azl3.0.20260602 | azl3.0.20260616 | | image-sync/oc-mirror | 690892d | 5bfc996 | | **clusters-service** | **6a49b32 (pinned — good)** | **— (b17f6fe held back)** |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bisecting the e2e-parallel regression on the automated image-digest PR #5789, which flipped from green to red at commit
128feb1.What
Bumps only the
hypershiftoperator image digest to the latest value from #5789, on top of currentmain. All other components (including ACM/MCE and everything else) stay at theirmaindigests.Why
At the pass→fail boundary (
7294d8c→128feb1) of #5789, only 6 digests changed: hypershift, ACM operator bundle, MCE bundle, OADP velero azure/hypershift plugins, and oc-mirror.clusters-servicedid not change. The failure signature isCreateHCPClustertiming out (clusters never provision) across ~16 tests, which points at hypershift/ACM. This PR isolates hypershift so e2e-parallel tells us whether the hypershift bump alone breaks provisioning.Testing
Relies on
ci/prow/e2e-parallel.make -C config/ detect-changeis clean (materialize idempotent). This is a diagnostic bisect PR — not intended to merge as-is.Special notes for your reviewer
Do not merge. Diagnostic bisect of #5789. Companion PRs isolate ACM and the remaining components.